预先接受的语言模型实现了最先进的导致各种自然语言处理（NLP）任务。 GPT-3表明，缩放预先训练的语言模型可以进一步利用它们的巨大潜力。最近提出了一个名为Ernie 3.0的统一框架，以预先培训大型知识增强型号，并培训了具有10亿参数的模型。 Ernie 3.0在各种NLP任务上表现出最先进的模型。为了探讨缩放的表现，我们培养了百卢比的3.0泰坦参数型号，在PaddlePaddle平台上有高达260亿参数的泰坦。此外，我们设计了一种自我监督的对抗性损失和可控语言建模损失，以使ERNIE 3.0 TITAN产生可信和可控的文本。为了减少计算开销和碳排放，我们向Ernie 3.0泰坦提出了一个在线蒸馏框架，教师模型将同时教授学生和培训。埃塞尼3.0泰坦是迄今为止最大的中国密集预训练模型。经验结果表明，Ernie 3.0泰坦在68个NLP数据集中优于最先进的模型。

三维（3D）医学图像的产生可能具有巨大的应用潜力，因为它考虑了3D解剖结构。但是，有两个问题可以防止有效培训3D医疗生成模型：（1）3D医学图像的获取和注释非常昂贵，导致培训图像不足，（2）大量参数是参与3D卷积。为了解决这两个问题，我们提出了一种名为3D Split＆Shuffle-Gan的新型GAN模型。为了解决3D数据稀缺问题，我们首先使用丰富的图像切片预先培训二维（2D）GAN模型，并夸大2D卷积权重以改善3D GAN的初始化。为GAN模型的生成器和鉴别器提出了新型的3D网络体系结构，以显着减少参数的数量，同时保持图像生成的质量。研究了许多体重通胀策略和参数有效的3D架构。对心脏（Stanford Aimi冠状动脉钙）和大脑（阿尔茨海默氏病神经成像计划）的实验表明，所提出的方法会导致改善的3D图像产生质量，参数较少。

视觉多层感知器（MLP）在计算机视觉任务中表现出了有希望的表现，并成为CNNS和Vision Transformers的主要竞争对手。他们使用令牌混合层来捕获交叉互动，而不是变形金刚使用的多头自我发项机制。然而，严重的参数化令牌混合层自然缺乏捕获局部信息和多粒性非本地关系的机制，因此它们的判别能力受到限制。为了解决这个问题，我们提出了一个新的位置空间门控单元（POSGU）。它利用经典相对位置编码（RPE）中使用的注意力公式，以有效地编码令牌混合的交叉关系。它可以成功地将视觉MLP的当前二次参数复杂度$ O（n^2）$ $ O（n^2）$ o（n）$（n）$和$ o（1）$。我们实验了两种RPE机制，并进一步提出了一个小组扩展，以实现多种环境的成就，以提高其表现力。然后，它们是一种新型视觉MLP的关键构建块，称为POSMLP。我们通过进行彻底的实验来评估所提出的方法的有效性，证明参数复杂性的提高或可比性能得到了改善或可比性。例如，对于在ImagEnet1k上训练的模型，我们实现了从72.14 \％\％\％\％的绩效提高，并且可学习的参数从$ 194M $ $ $ $ $ $ $ $ 1.182亿美元。代码可以在\ href {https://github.com/zhicaiwww/posmlp} {https://github.com/zhicaiwww/posmlp}中找到代码。

视频变压器自然会产生比静态视觉变压器更重的计算负担，因为前者在二次复杂性$（t^2n^2）$的当前关注下，$ t $ t $倍的序列比后者长。现有作品将颞轴视为空间轴的简单扩展，重点是通过通用池或局部窗口缩短时空序列，而无需使用时间冗余。但是，视频自然包含相邻框架之间的冗余信息；因此，我们可能会以扩张的方式抑制视觉上相似帧的注意力。基于这一假设，我们提出了圈，长期`\ textbf {\ textit {leap coasitive}}''（la），短期`\ textbf {\ textbf {\ textit {preckentien shiftit {presentiic shift}}}'（'（ \ textit {p} -Shift）用于视频变压器的模块，带有$（2Tn^2）$复杂性。具体而言，``la''将长期帧分为对，然后通过注意来重构每个离散对。 ``\ textit {p} -shift''在时间邻居之间交换特征，以面对短期动力学的丧失。通过用圈替换香草2D的注意，我们可以将静态变压器调整为视频，其中零额外的参数和可忽视的计算开销（$ \ sim $ 2.6 \％）。对标准动力学-400基准的实验表明，我们的圈量变压器可以在CNN和Transformer Sotas之间的准确性，FLOP和参数方面实现竞争性能。我们以\ sloppy \ href {https://github.com/videonetworks/laps-transformer} {\ textit {\ textit {\ color {agenta} {https://github.com/videonetworks/laps-transsfornss/laps-transformer} { }}。

Recent years have witnessed the rapid growth of Small Private Online Courses (SPOC) which is able to highly customized and personalized to adapt variable educational requests, in which machine learning techniques are explored to summarize and predict the learner's performance, mostly focus on the final grade. However, the problem is that the final grade of learners on SPOC is generally seriously imbalance which handicaps the training of prediction model. To solve this problem, a sampling batch normalization embedded deep neural network (SBNEDNN) method is developed in this paper. First, a combined indicator is defined to measure the distribution of the data, then a rule is established to guide the sampling process. Second, the batch normalization (BN) modified layers are embedded into full connected neural network to solve the data imbalanced problem. Experimental results with other three deep learning methods demonstrates the superiority of the proposed method.

Compressed videos often exhibit visually annoying artifacts, known as Perceivable Encoding Artifacts (PEAs), which dramatically degrade video visual quality. Subjective and objective measures capable of identifying and quantifying various types of PEAs are critical in improving visual quality. In this paper, we investigate the influence of four spatial PEAs (i.e. blurring, blocking, bleeding, and ringing) and two temporal PEAs (i.e. flickering and floating) on video quality. For spatial artifacts, we propose a visual saliency model with a low computational cost and higher consistency with human visual perception. In terms of temporal artifacts, self-attention based TimeSFormer is improved to detect temporal artifacts. Based on the six types of PEAs, a quality metric called Saliency-Aware Spatio-Temporal Artifacts Measurement (SSTAM) is proposed. Experimental results demonstrate that the proposed method outperforms state-of-the-art metrics. We believe that SSTAM will be beneficial for optimizing video coding techniques.

As one of the most important psychic stress reactions, micro-expressions (MEs), are spontaneous and transient facial expressions that can reveal the genuine emotions of human beings. Thus, recognizing MEs (MER) automatically is becoming increasingly crucial in the field of affective computing, and provides essential technical support in lie detection, psychological analysis and other areas. However, the lack of abundant ME data seriously restricts the development of cutting-edge data-driven MER models. Despite the recent efforts of several spontaneous ME datasets to alleviate this problem, it is still a tiny amount of work. To solve the problem of ME data hunger, we construct a dynamic spontaneous ME dataset with the largest current ME data scale, called DFME (Dynamic Facial Micro-expressions), which includes 7,526 well-labeled ME videos induced by 671 participants and annotated by more than 20 annotators throughout three years. Afterwards, we adopt four classical spatiotemporal feature learning models on DFME to perform MER experiments to objectively verify the validity of DFME dataset. In addition, we explore different solutions to the class imbalance and key-frame sequence sampling problems in dynamic MER respectively on DFME, so as to provide a valuable reference for future research. The comprehensive experimental results show that our DFME dataset can facilitate the research of automatic MER, and provide a new benchmark for MER. DFME will be published via https://mea-lab-421.github.io.

Face Anti-spoofing (FAS) is essential to secure face recognition systems from various physical attacks. However, recent research generally focuses on short-distance applications (i.e., phone unlocking) while lacking consideration of long-distance scenes (i.e., surveillance security checks). In order to promote relevant research and fill this gap in the community, we collect a large-scale Surveillance High-Fidelity Mask (SuHiFiMask) dataset captured under 40 surveillance scenes, which has 101 subjects from different age groups with 232 3D attacks (high-fidelity masks), 200 2D attacks (posters, portraits, and screens), and 2 adversarial attacks. In this scene, low image resolution and noise interference are new challenges faced in surveillance FAS. Together with the SuHiFiMask dataset, we propose a Contrastive Quality-Invariance Learning (CQIL) network to alleviate the performance degradation caused by image quality from three aspects: (1) An Image Quality Variable module (IQV) is introduced to recover image information associated with discrimination by combining the super-resolution network. (2) Using generated sample pairs to simulate quality variance distributions to help contrastive learning strategies obtain robust feature representation under quality variation. (3) A Separate Quality Network (SQN) is designed to learn discriminative features independent of image quality. Finally, a large number of experiments verify the quality of the SuHiFiMask dataset and the superiority of the proposed CQIL.

Interview has been regarded as one of the most crucial step for recruitment. To fully prepare for the interview with the recruiters, job seekers usually practice with mock interviews between each other. However, such a mock interview with peers is generally far away from the real interview experience: the mock interviewers are not guaranteed to be professional and are not likely to behave like a real interviewer. Due to the rapid growth of online recruitment in recent years, recruiters tend to have online interviews, which makes it possible to collect real interview data from real interviewers. In this paper, we propose a novel application named EZInterviewer, which aims to learn from the online interview data and provides mock interview services to the job seekers. The task is challenging in two ways: (1) the interview data are now available but still of low-resource; (2) to generate meaningful and relevant interview dialogs requires thorough understanding of both resumes and job descriptions. To address the low-resource challenge, EZInterviewer is trained on a very small set of interview dialogs. The key idea is to reduce the number of parameters that rely on interview dialogs by disentangling the knowledge selector and dialog generator so that most parameters can be trained with ungrounded dialogs as well as the resume data that are not low-resource. Evaluation results on a real-world job interview dialog dataset indicate that we achieve promising results to generate mock interviews. With the help of EZInterviewer, we hope to make mock interview practice become easier for job seekers.

Nowadays, time-stamped web documents related to a general news query floods spread throughout the Internet, and timeline summarization targets concisely summarizing the evolution trajectory of events along the timeline. Unlike traditional document summarization, timeline summarization needs to model the time series information of the input events and summarize important events in chronological order. To tackle this challenge, in this paper, we propose a Unified Timeline Summarizer (UTS) that can generate abstractive and extractive timeline summaries in time order. Concretely, in the encoder part, we propose a graph-based event encoder that relates multiple events according to their content dependency and learns a global representation of each event. In the decoder part, to ensure the chronological order of the abstractive summary, we propose to extract the feature of event-level attention in its generation process with sequential information remained and use it to simulate the evolutionary attention of the ground truth summary. The event-level attention can also be used to assist in extracting summary, where the extracted summary also comes in time sequence. We augment the previous Chinese large-scale timeline summarization dataset and collect a new English timeline dataset. Extensive experiments conducted on these datasets and on the out-of-domain Timeline 17 dataset show that UTS achieves state-of-the-art performance in terms of both automatic and human evaluations.